1. Technical Field
The present invention relates to a multi-lens camera system capable of acquiring parallax information and a vehicle mounting the multi-lens camera system, and a range-finding method executed by the multi-lens camera system.
2. Related Art
Collision avoidance systems involving the use of in-vehicle stereo cameras have become widespread. A stereoscopic image of the area in front of the vehicle is generated using the stereo cameras, and an obstacle is detected and a distance to the obstacle is measured based on the generated stereoscopic image. The driver can then be alerted to take corrective action to avoid a collision or maintain a safe minimum distance between vehicles. Alternatively, the system can engage a control device such as the brakes and the steering.
Further, with improving sensor performance such as higher image resolutions, the focus of vehicle environment sensing has shifted from highway driving to city driving. In the city, the target sensing environment is much more diversified compared to the highway. The biggest problem for recognition processing in a complex sensing environment is that such diversity complicates processing, as a result of which the processing time lengthens and object misrecognition increases.
In recognition processing in the conventional in-vehicle stereo camera, initially, edges are detected over substantially the entire captured image and parallax is calculated for the edge-detected pixel position. Then, by executing clustering using the parallax information which is the calculation result and adding various types of information to the clustering result, the recognition target is finally detected.
FIG. 9 is a flow chart illustrating a general process of object recognition using parallax information. Herein, the recognition targets include a vehicle in front, a pedestrian, a motorbike, a bicycle, a traffic sign, and a traffic light as an object on a road. Initially, using a stereo camera including a reference camera unit and a comparison camera unit like that shown in FIG. 1A, a captured image (stereo image) including a reference image and a comparison image is acquired at step S201. Then, using luminance information in the reference image, a dividing line (e.g., white lines and yellow lines, including stop line) on the road is recognized, at step S208. Along with this process, parallax is calculated based on the reference image and the comparison image at step S202, clustering is executed using the parallax information at step S203, and the clustering result is modified using the size of object, such as the vehicle, the pedestrian, the motorbike, the bicycle, the traffic sign, and the traffic light, at step S204.
FIG. 10 is an object-recognized image obtained by the processes executed at steps S201 through S204. In FIG. 10, reference numeral 100 represents the clustering result recognized as the vehicle object size, 101 represents the clustering result recognized as the pedestrian, the motorbike, and bicycle size, and 102 represents the clustering result recognized as the traffic sign and the traffic light size. As is clear from the object-recognized image of FIG. 10, in particular, since the traffic sign and the traffic light are small, they are often misrecognized, as indicated by frames 102′. In addition, misrecognition of a pedestrian is also seen in an upper region of the image in FIG. 10, indicated by frame 101′.
Various conditional branch processes are executed on these object misrecognitions at steps S205 and later shown in FIG. 9. For example, at step S205, a height of the road surface is calculated based on the dividing line (stop line) recognized at step S208, and the clustering result is modified using target model information for the pedestrian and the fallen object. In addition, at step S206, based on the luminance information in the reference image, a final determination is made regarding the target object for the region for which clustering is executed, using adaptive boosting (AdaBoost). Finally, the three-dimensional position of the recognized object is output at step S207.
In the processes of from steps S201 through S204, if serious object misrecognition occurs or the target object cannot be separately recognized successfully, many complicated processes arise, for example, various conditional branch processing is needed subsequent recognition processing, or previous-stage processing must be revisited. Accordingly, it is important to improve the recognition success rate of the parallax calculation and the clustering, and to minimize misrecognition.
In order to recognize the objects in the images, for example, JP-2004-173195-A proposes a stereo camera system that captures scenery around the vehicle, calculates range data representing two-dimensional distribution of the distance between a camera that outputs color images and a target in a monitoring region, and recognizes another vehicle driving in front in the same lane or in an adjacent lane. Herein, a winker region having a predetermined dimension is set based on the vehicle position in the target-recognized image and a pixel constituting a color component of the winker is detected based on the color image. Accordingly, the winker and the near vehicle can be recognized simultaneously, using the color information and the parallax information. With this configuration, using a combination of the color information and the parallax information, multiple objects can be recognized accurately and simultaneously.
In the above-described method, as for the recognition objects whose color information is known in advance, by detecting the color components of the recognition objects based on the color image acquired by the color camera, the object can be detected accurately. However, it is necessary to provide the color camera in addition to the stereo camera, set the region where the color object is recognized in the image, and perform additional recognition processing using a different algorithm. In addition, in order to calculate the parallax in the entire image and the parallax in the setting region respectively, switching the parallax calculation algorithm and parameter is required. This operation complicates calculation processing and cannot solve the above-described problem.